There has been a demonstrated widespread need for a better process to obtain three-dimensional measurement and high resolution imagery for inspection and documentation of structural features that are dangerous to access directly. Very large investments continue to be made to acquire oblique image archives that capture imagery over broad areas with relatively limited resolution, redundancy, and accuracy compared to the imagery attainable through the use of UAVs. Previously disclosed methods for UAV image and data acquisition conducive to three-dimensional modeling and efficient inspection lack adequate simplicity, reliability, and affordability for common usage by untrained operators.
Ground based survey techniques by contrast are notoriously slow, prone to obstructions, and often require acquisition expertise and deployment of expensive sensors. Previous aerial Computer Aided Design (CAD) structural modeling approaches that rely on image edge detection to delineate facet seams are prone to fail with shadows, low contrast lighting, low texture building material, gutters or subtle slope changes. Structure modeling from three-dimensional point clouds fail where point density or accuracy is inadequate to cover needed surface facets with sufficient redundancy.
For a variety of purposes, the condition of building exteriors regularly needs to be assessed to evaluate and to permit cost effective maintenance and repair. Likewise, detailed structure measurement is required prior to repair or renovation. Architectural design plans, if available, are often an insufficient measurement source due to the need for reliable and current as-built dimensional data.
The construction and insurance industries expend substantial time and effort in performing both inspection and measurement of structures. Experienced field workers often physically survey the structure and document the condition, scope, and cost of repairs. This assessment and estimation work is costly, time sensitive, and often dangerous. The resulting documentation is incomplete, subjective, and prone to dispute after the work is completed.
These problems are particularly pronounced for portions of a property that are difficult to observe due to the size, height, slope, and/or location of the structure. Repairs are often urgently needed after catastrophes or severe weather. Time pressures to complete those repairs can further increase the likelihood of delays, errors, and fraud. For example, many residential roofs will need to be inspected and measured by both roofers and insurance adjusters after a hailstorm. For steep roofs, safety concerns often require deployment of multiple field workers who typically collect and document measurements with tape measures and hand sketches. Roof condition is documented by a jumble of phone photos and surface chalk markings at precarious locations.
Installation of roof top solar arrays also require measurements of roof facets as well as protrusions such vent caps, vent pipes, vent stacks, antennae, and skylights that constrain array placement. Estimators climb up on roofs to subjectively assess where the arrays can be placed, the pitch and orientation of each facet with respect to the sun, and how much sunlight will be shadowed by nearby trees, air conditioning units, or neighboring buildings. Often a photo mosaic is captured by the technician from on top of the roof for a visual record of the roof area, but this perspective is highly obstructed compared to aerial photos. CAD modeling systems are available to precisely design solar array layout and forecast electrical yield of an array based on this information but the as-built CAD geometry including detailed protrusion locations and tree models are not readily obtainable from an efficient data collection and information extraction process.
Aerial photography. As the use of aerial-captured imagery and design models has become more prevalent in the insurance and construction industries, the associated deliverables result in the need for information that is more current, accurate, broadly available, and readily available to manage and repair structures of interest. Various features of a property critical for accurate construction or repair estimates are often not visible in aerial imagery archives because of limited coverage, inadequate resolution, occlusions from trees or roof overhangs, and out dated content. Limitations of resolution and camera perspectives also impact the detail, precision, and completeness of automatically obtainable three-dimensional measurement from these sparsely captured aerial archives which in turn leads to subjective and highly manual sketching techniques described in the prior art for design modeling.
The shortcomings of maintaining regional aerial image archives become prevalent with the increasing costs of capturing and delivering large amounts of imagery, much of which is unusable for assessing the actual condition of structures and determining the accurate repair cost parameters. As users expect higher resolution with greater detail, dated or imprecise images are not easily corrected and translated into workable models for estimation since they are typically flown years in advance of damaging events over large areas.
Ground surveys. Even with expertly captured supplemental ground level photos, it is often time prohibitive or impossible to completely capture a structure's exterior because the images needed for thorough inspection and automated measurement cannot be obtained due to vegetation occlusions, structural self-occlusions, property boundaries, terrain, or the breadth of the property.
Structured light sensors such as Microsoft Kinect and Google Tango do not work at long range or in direct sunlight. Terrestrial tripod mounted laser scanners have been used with increasing popularity among professional surveyors over the last decade and provide centimeter level precision but this approach demands hours of acquisition and processing effort by trained technicians. Furthermore, this technique typically fails to capture the entire structural exterior, especially roofs, due to obstructions, difficulty in obtaining appropriate observation angles, hazards, and time required on site. This is especially true for structures that are built on hillsides, closely adjacent to other structures, or that are surrounded by shrubs, trees, or fencing.
Other UAV techniques. Some UAV mapping surveys may be performed from a series of downward pointing (nadir) photos captured in a linear grid pattern over an area of interest requiring a series of inefficient U-turn maneuvers and offering little or no overlap between the initial and final photos in the series. Tree tops severely disrupt image matching at low altitude and image collections that do not maintain persistent oblique focus on the structure to be measured will yield reduced accuracy because large groups of photos will not be usable to reconstruct the structure completely. Overlap between photos can become irregular if the photos are not captured precisely where and when planned from such piecewise linear trajectories. These inherent flaws yield reduced measurement accuracy—large groups of photos fail to provide a basis for the complete and consistent reconstruction of a structure. Furthermore, these techniques demand professional quality aircraft hardware to ensure precise aircraft positioning, attitude sensing, and camera stability especially in high wind or low light conditions.
Ground based GPS/GNSS correction with costly and heavy receivers may be needed to correct time varying errors. Cheap and lightweight rolling shutter cameras are not typically by the aerial survey community because of distortions introduced during photogrammetric matching from low redundancy photo collections. Even middle grade consumer camera systems are often blamed for inadequate photogrammetric reconstruction when in fact results could actually be improved with better image collection techniques.
Multi directional oblique camera rigs that are often used for three-dimensional reconstruction of structural facades are prohibitively heavy for affordable and safe micro UAVs. Nadir photos even from wide angle lens do not capture facades redundantly enough for reliable reconstruction. Oblique image coverage with UAVs has occasionally been demonstrated with ad hoc, unreliable, time-consuming manually steered acquisition procedures that are not generic or simple enough for casual field operators to use regularly.
Excessive data collection increases data transfer and management costs and reduces efficiency of remote inspection. High resolution oblique photos captured without a means to precisely mask out the unintended coverage of neighboring properties prior to distribution raises the possibility of privacy actions. More sophisticated UAV acquisition strategies demand either scarce a priori or compute intensive real-time three-dimensional models or do not ensure image collections are optimally designed for automated reconstruction and contains adequate context for easy visual inspection.
The three-dimensional reconstruction of structural surfaces captured with nadir or inadequate oblique imagery will contain conspicuous voids especially for lightly textured low contrast surfaces or surfaces that are partially obstructed by trees which makes automated three-dimensional vector modeling extremely challenging.
Aerial LIDAR. Much prior research has been devoted to modeling structures from LIDAR point clouds partly because laser measurements are quite precise from high altitude flights when captured with high cost sensor and inertial measurement systems. Automated point cloud clustering techniques have been proposed to convert sparse precise LIDAR data into structural point cloud models but fail to capture details of a meter or less in size due to limited point density (less than twenty points per meter).
Heavy high frequency LIDAR sensors can penetrate vegetation and reduce obstructions from trees compared to photogrammetric reconstruction obtained from a sparse photo acquisition. However, LIDAR systems cost over ten times more and are five times heavier than camera-only micro UAV systems that are manufactured in the millions for the consumer drone market. Furthermore, LIDAR systems cannot get resolution of ground-angle views of structures or areas of interest.
The lightest and most affordable LIDARs lack sufficient power or collection speed to make rapid and reliable data collection possible. The added cost, weight, and energy demands makes LIDAR infeasible for on demand field measurements of specific properties. Even if LIDAR was practical to use for measurement, camera data would still need to be collected as well for manual or automated visual inspection.